Safety Verification of Continuous-Space Pure Jump Markov Processes
نویسندگان
چکیده
We study the probabilistic safety verification problem for pure jumpMarkov processes, a class of models that generalizes continuous-time Markov chains over continuous (uncountable) state spaces. Solutions of these processes are piecewise constant, right-continuous functions from time to states. Their jump (or reset) times are realizations of a Poisson process, characterized by a jump rate function that can be both timeand state-dependent. Upon jumping in time, the new state of the solution process is specified according to a (continuous) stochastic conditional kernel. After providing a full characterization of safety properties of these processes, we describe a formal method to abstract the process as a finitestate discrete-time Markov chain; this approach is formal in that it provides a-priori error bounds on the precision of the abstraction, based on the continuity properties of the stochastic kernel of the process and of its jump rate function. We illustrate the approach on a case study of thermostatically controlled loads.
منابع مشابه
MDP algorithms for portfolio optimization problems in pure jump markets
We consider the problem of maximizing the expected utility of the terminal wealth of a portfolio in a continuous-time pure jump market with general utility function. This leads to an optimal control problem for Piecewise Deterministic Markov Processes. Using an embedding procedure we solve the problem by looking at a discrete-time contracting Markov Decision Process. Our aim is to show that thi...
متن کاملLumpability and Commutativity of Markov Processes
We introduce the concepts of lumpability and commutativity of a continuous time discrete state space Markov process, and provide a necessary and sufficient condition for a lumpable Markov process to be commutative. Under suitable conditions we recover some of the basic quantities of the original Markov process from the jump chain of the lumped Markov process.
متن کاملMarkov chain Monte Carlo for continuous-time discrete-state systems
A variety of phenomena are best described using dynamical models which operate on a discrete state space and in continuous time. Examples include Markov (and semiMarkov) jump processes, continuous-time Bayesian networks, renewal processes and other point processes. These continuous-time, discrete-state models are ideal building blocks for Bayesian models in fields such as systems biology, genet...
متن کاملOn $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes with general state spaces. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of quadratic stochastic processes.
متن کاملBounds on the L2 Spectrum for Markov Chains and Markov Processes: a Generalization of Cheeger's Inequality
We prove a general version of Cheeger's inequality for discretetime Markov chains and continuous-time Markovian jump processes, both reversible and nonreversible, with general state space. We also prove a version of Cheeger's inequality for Markov chains and processes with killing. As an application, we prove L2 exponential convergence to equilibrium for random walk with inward drift on a class...
متن کامل